{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 4: Automated ML pipeline\n",
    "\n",
    "MLRun Project is a container for all your work on a particular activity: all of the associated code, functions, jobs/workflows and artifacts. Projects can be mapped to `git` repositories, which enable versioning, collaboration, and CI/CD.\n",
    "Users can create project definitions using the SDK or a yaml file and store those in MLRun DB, file, or archive.\n",
    "Once the project is loaded you can run jobs/workflows that refer to any project element by name, allowing separation between configuration and code. \n",
    "\n",
    "Projects contain `workflows` that execute the registered functions in a sequence/graph (DAG), can reference project \n",
    "parameters, secrets and artifacts by name. This notebook demonstrates how to build an automated workflow with \n",
    "**feature selection**, **training**, **testing**, and **deployment**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-prerequisites\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: Setting up your project\n",
    "\n",
    "To run a pipeline, you first need to get or create a project object and define/import the required functions for its execution.\n",
    "See the [**Create, save, and use projects**](../../projects/create-project.md) for details.\n",
    "\n",
    "The following code gets or creates a user project named \"fraud-demo<username>\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the base project name\n",
    "project_name = \"fraud-demo\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-02-15 14:52:09,517 [info] loaded project fraud-demo from MLRun DB\n"
     ]
    }
   ],
   "source": [
    "import mlrun\n",
    "\n",
    "# Initialize the MLRun project object\n",
    "project = mlrun.get_or_create_project(project_name, context=\"./\", user_project=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-step-import-functions\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Updating project and function definitions\n",
    "\n",
    "You need to save the definitions for the function you use in the projects. This enables automatically converting code \n",
    "to functions or import external functions whenever you load new versions of your code or when you run automated CI/CD \n",
    "workflows. In addition, you may want to set other project attributes such as global parameters, secrets, and data.\n",
    "\n",
    "Your code can be stored in Python files, notebooks, external repositories, packaged containers, etc. You use the \n",
    "`project.set_function()` method to register your code in the project. The definitions are saved to the project object, as \n",
    "well as in a YAML file in the root of our project.\n",
    "Functions can also be imported from MLRun marketplace (using the `hub://` schema).\n",
    "\n",
    "This tutorial uses these functions:\n",
    "- `feature_selection` &mdash; the first function, which determines the top features to be used for training.\n",
    "- `train` &mdash; the model-training function\n",
    "- `evaluate` &mdash; the model-testing function\n",
    "- `mlrun-model` &mdash; the model-serving function\n",
    "\n",
    "```{admonition} Note\n",
    "`set_function` uses the `code_to_function` and `import_function` methods under the hood (used in the previous notebooks), but in addition it saves the function configurations in the project spec for use in automated workflows and CI/CD. \n",
    "```\n",
    "Add the function definitions to the project along with parameters and data artifacts and save the project."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-view-project-functions\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Names with underscore '_' are about to be deprecated, use dashes '-' instead. Replacing underscores with dashes.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<mlrun.runtimes.serving.ServingRuntime at 0x7f5701e79520>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "project.set_function(\"hub://feature_selection\", \"feature_selection\")\n",
    "project.set_function(\"hub://auto_trainer\", \"train\")\n",
    "project.set_function(\"hub://v2_model_server\", \"serving\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<mlrun.projects.project.MlrunProject at 0x7f5720229280>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# set project level parameters and save\n",
    "project.spec.params = {\"label_column\": \"label\"}\n",
    "project.save()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>When you save the project it stores the project definitions in the `project.yaml`. This allows you to load the project \n",
    "from the source control (GIT) and run it with a single command or API call.\n",
    "\n",
    "The project YAML for this project can be printed using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kind: project\n",
      "metadata:\n",
      "  name: fraud-demo-dani\n",
      "  created: '2023-02-15T14:40:29.807000'\n",
      "spec:\n",
      "  params:\n",
      "    label_column: label\n",
      "  functions:\n",
      "  - url: hub://feature_selection\n",
      "    name: feature_selection\n",
      "  - url: hub://auto_trainer\n",
      "    name: train\n",
      "  - url: hub://v2_model_server\n",
      "    name: serving\n",
      "  workflows: []\n",
      "  artifacts: []\n",
      "  source: ''\n",
      "  desired_state: online\n",
      "  owner: dani\n",
      "status:\n",
      "  state: online\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(project.to_yaml())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving and loading projects from GIT\n",
    "\n",
    "After you saved your project and its elements (functions, workflows, artifacts, etc.) you can commit all your changes to a \n",
    "GIT repository. This can be done using standard GIT tools or using MLRun `project` methods such as `pull`, `push`, \n",
    "`remote`, which calls the Git API for you.\n",
    "\n",
    "Projects can then be loaded from Git using MLRun `load_project` method, for example: \n",
    "\n",
    "    project = mlrun.load_project(\"./myproj\", \"git://github.com/mlrun/project-demo.git\", name=project_name)\n",
    "    \n",
    "or using MLRun CLI:\n",
    "\n",
    "    mlrun project -n myproj -u \"git://github.com/mlrun/project-demo.git\" ./myproj\n",
    "    \n",
    "Read [CI/CD integration](../../projects/ci-integration.md) for more details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-kubeflow-pipelines\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Kubeflow pipelines\n",
    "\n",
    "You're now ready to create a full ML pipeline.\n",
    "This is done by using [Kubeflow Pipelines](https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview/) &mdash;\n",
    "an open-source framework for building and deploying portable, scalable machine-learning workflows based on Docker containers.\n",
    "MLRun leverages this framework to take your existing code and deploy it as steps in the pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Defining and saving a pipeline workflow\n",
    "\n",
    "A pipeline is created by running an MLRun **\"workflow\"**.\n",
    "The following code defines a workflow and writes it to a file in your local directory.\n",
    "(The file name is **workflow.py**.)\n",
    "The workflow describes a directed acyclic graph (DAG) for execution using Kubeflow Pipelines, and depicts the connections \n",
    "between the functions and the data as part of an end-to-end pipeline.\n",
    "The workflow file has a definition of a pipeline DSL for connecting the function inputs and outputs.\n",
    "\n",
    "The defined pipeline includes the following steps:\n",
    "\n",
    "- Perform feature selection (`feature_selection`).\n",
    "- Train and the model (`train`).\n",
    "- Test the model with its test data set (`evaluate`).\n",
    "- Deploy the model as a real-time serverless function (`deploy`).\n",
    "\n",
    "```{admonition} Note\n",
    "A pipeline can also include continuous build integration and deployment (CI/CD) steps, such as building container images and deploying models.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing workflow.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile workflow.py\n",
    "import mlrun\n",
    "from kfp import dsl\n",
    "from mlrun.model import HyperParamOptions\n",
    "    \n",
    "@dsl.pipeline(\n",
    "    name=\"Fraud Detection Pipeline\",\n",
    "    description=\"Detecting fraud from a transactions dataset\"\n",
    ")\n",
    "\n",
    "def kfpipeline(vector_name='transactions-fraud'):\n",
    "    \n",
    "    project = mlrun.get_current_project()\n",
    "    \n",
    "    # Feature selection   \n",
    "    feature_selection = run_function(\n",
    "        \"feature_selection\",\n",
    "        name=\"feature_selection\",\n",
    "        params={'output_vector_name': \"short\",\n",
    "                \"label_column\": project.get_param('label_column', 'label'),\n",
    "                \"k\": 18,\n",
    "                \"min_votes\": 2,\n",
    "                'ignore_type_errors': True\n",
    "               },\n",
    "        inputs={'df_artifact': project.get_artifact_uri(vector_name, 'feature-vector')},\n",
    "        outputs=['feature_scores', 'selected_features_count', 'top_features_vector', 'selected_features'])\n",
    "    \n",
    "    \n",
    "    # train with hyper-paremeters\n",
    "    train = run_function(\n",
    "        \"train\",\n",
    "        name=\"train\",\n",
    "        handler=\"train\",\n",
    "        params={\"sample\": -1, \n",
    "                \"label_column\": project.get_param('label_column', 'label'),\n",
    "                \"test_size\": 0.10},\n",
    "        hyperparams={\"model_name\": ['transaction_fraud_rf', \n",
    "                                    'transaction_fraud_xgboost', \n",
    "                                    'transaction_fraud_adaboost'],\n",
    "                     'model_class': [\"sklearn.ensemble.RandomForestClassifier\", \n",
    "                                     \"sklearn.linear_model.LogisticRegression\",\n",
    "                                     \"sklearn.ensemble.AdaBoostClassifier\"]},\n",
    "        hyper_param_options=HyperParamOptions(selector=\"max.accuracy\"),\n",
    "        inputs={\"dataset\": feature_selection.outputs['top_features_vector']},\n",
    "        outputs=['model', 'test_set'])\n",
    "    \n",
    "            \n",
    "    # test and visualize your model\n",
    "    test = run_function(\n",
    "        \"train\",\n",
    "        name=\"evaluate\",\n",
    "        handler='evaluate',\n",
    "        params={\"label_columns\": project.get_param('label_column', 'label'),\n",
    "                \"model\": train.outputs[\"model\"], \n",
    "                \"drop_columns\": project.get_param('label_column', 'label')},\n",
    "        inputs={\"dataset\": train.outputs[\"test_set\"]})\n",
    "    \n",
    "    # route your serving model to use enrichment\n",
    "    funcs['serving'].set_topology('router', \n",
    "                                  'mlrun.serving.routers.EnrichmentModelRouter', \n",
    "                                  name='EnrichmentModelRouter', \n",
    "                                  feature_vector_uri=\"transactions-fraud-short\", \n",
    "                                  impute_policy={\"*\": \"$mean\"},\n",
    "                                  exist_ok=True)\n",
    "\n",
    "    \n",
    "    # deploy your model as a serverless function, you can pass a list of models to serve \n",
    "    deploy = deploy_function(\"serving\", models=[{\"key\": 'fraud', \"model_path\": train.outputs[\"model\"]}])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-step-register-workflow\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Registering the workflow\n",
    "\n",
    "Use the `set_workflow` MLRun project method to register your workflow with MLRun.\n",
    "The following code sets the `name` parameter to the selected workflow name (\"main\") and the `code` parameter to the name of \n",
    "the workflow file that is found in your project directory (**workflow.py**)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the workflow file as \"main\"\n",
    "project.set_workflow(\"main\", \"workflow.py\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-step-run-pipeline\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Running a pipeline\n",
    "\n",
    "First run the following code to save your project:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<mlrun.projects.project.MlrunProject at 0x7f5720229280>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "project.save()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `run` MLRun project method to execute your workflow pipeline with Kubeflow Pipelines.\n",
    "\n",
    "You can pass **`arguments`** or set the **`artifact_path`** to specify a unique path for storing the workflow artifacts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>Pipeline running (id=2e7556a2-c398-4134-8229-163bd7ee3ec3), <a href=\"https://dashboard.default-tenant.app.vmdev94.lab.iguazeng.com/mlprojects/fraud-demo-dani/jobs/monitor-workflows/workflow/2e7556a2-c398-4134-8229-163bd7ee3ec3\" target=\"_blank\"><b>click here</b></a> to view the details in MLRun UI</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.43.0 (0)\n",
       " -->\n",
       "<!-- Title: kfp Pages: 1 -->\n",
       "<svg width=\"248pt\" height=\"188pt\"\n",
       " viewBox=\"0.00 0.00 248.05 188.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 184)\">\n",
       "<title>kfp</title>\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-184 244.05,-184 244.05,4 -4,4\"/>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1661087637 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1661087637</title>\n",
       "<ellipse fill=\"green\" stroke=\"black\" cx=\"114.05\" cy=\"-162\" rx=\"89.08\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"114.05\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\">feature&#45;selection</text>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878</title>\n",
       "<ellipse fill=\"green\" stroke=\"black\" cx=\"114.05\" cy=\"-90\" rx=\"33.29\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"114.05\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\">train</text>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1661087637&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1661087637&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M114.05,-143.7C114.05,-135.98 114.05,-126.71 114.05,-118.11\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"117.55,-118.1 114.05,-108.1 110.55,-118.1 117.55,-118.1\"/>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1793851311 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1793851311</title>\n",
       "<ellipse fill=\"green\" stroke=\"black\" cx=\"50.05\" cy=\"-18\" rx=\"50.09\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"50.05\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">evaluate</text>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1793851311 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1793851311</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M99.84,-73.46C91.63,-64.48 81.11,-52.98 71.87,-42.87\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"74.43,-40.49 65.1,-35.47 69.27,-45.21 74.43,-40.49\"/>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1984023578 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1984023578</title>\n",
       "<polygon fill=\"green\" stroke=\"black\" points=\"240.05,-36 122.05,-36 118.05,-32 118.05,0 236.05,0 240.05,-4 240.05,-36\"/>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"236.05,-32 118.05,-32 \"/>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"236.05,-32 236.05,0 \"/>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"236.05,-32 240.05,-36 \"/>\n",
       "<text text-anchor=\"middle\" x=\"179.05\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">deploy&#45;serving</text>\n",
       "</g>\n",
       "<!-- fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1984023578 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;2083391878&#45;&gt;fraud&#45;detection&#45;pipeline&#45;kwzvs&#45;1984023578</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M128.47,-73.46C136.5,-64.82 146.68,-53.85 155.81,-44.03\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"158.59,-46.17 162.83,-36.46 153.46,-41.41 158.59,-46.17\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x7f5764d051f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h2>Run Results</h2><h3>[info] Workflow 2e7556a2-c398-4134-8229-163bd7ee3ec3 finished, state=Succeeded</h3><br>click the hyper links below to see detailed results<br><table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>uid</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td><div title=\"f5ea23f6f2ac4e37be7fc0e5f010d26c\"><a href=\"https://dashboard.default-tenant.app.vmdev94.lab.iguazeng.com/mlprojects/fraud-demo-dani/jobs/monitor/f5ea23f6f2ac4e37be7fc0e5f010d26c/overview\" target=\"_blank\" >...f010d26c</a></div></td>\n",
       "      <td>Feb 15 14:53:55</td>\n",
       "      <td>completed</td>\n",
       "      <td>evaluate</td>\n",
       "      <td><div class=\"dictlist\">label_columns=label</div><div class=\"dictlist\">model=store://artifacts/fraud-demo-dani/transaction_fraud_adaboost:2e7556a2-c398-4134-8229-163bd7ee3ec3</div><div class=\"dictlist\">drop_columns=label</div></td>\n",
       "      <td><div class=\"dictlist\">evaluation_accuracy=0.991504247876062</div><div class=\"dictlist\">evaluation_f1_score=0.4137931034482759</div><div class=\"dictlist\">evaluation_precision_score=0.42857142857142855</div><div class=\"dictlist\">evaluation_recall_score=0.4</div></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td><div title=\"7a77e271892a4dd1a738965e3c58eeba\"><a href=\"https://dashboard.default-tenant.app.vmdev94.lab.iguazeng.com/mlprojects/fraud-demo-dani/jobs/monitor/7a77e271892a4dd1a738965e3c58eeba/overview\" target=\"_blank\" >...3c58eeba</a></div></td>\n",
       "      <td>Feb 15 14:53:00</td>\n",
       "      <td>completed</td>\n",
       "      <td>train</td>\n",
       "      <td><div class=\"dictlist\">sample=-1</div><div class=\"dictlist\">label_column=label</div><div class=\"dictlist\">test_size=0.1</div></td>\n",
       "      <td><div class=\"dictlist\">best_iteration=9</div><div class=\"dictlist\">accuracy=0.991504247876062</div><div class=\"dictlist\">f1_score=0.4137931034482759</div><div class=\"dictlist\">precision_score=0.42857142857142855</div><div class=\"dictlist\">recall_score=0.4</div></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td><div title=\"3f91fa8cabe24efc81c48736d5f7ba87\"><a href=\"https://dashboard.default-tenant.app.vmdev94.lab.iguazeng.com/mlprojects/fraud-demo-dani/jobs/monitor/3f91fa8cabe24efc81c48736d5f7ba87/overview\" target=\"_blank\" >...d5f7ba87</a></div></td>\n",
       "      <td>Feb 15 14:52:23</td>\n",
       "      <td>completed</td>\n",
       "      <td>feature_selection</td>\n",
       "      <td><div class=\"dictlist\">output_vector_name=short</div><div class=\"dictlist\">label_column=label</div><div class=\"dictlist\">k=18</div><div class=\"dictlist\">min_votes=2</div><div class=\"dictlist\">ignore_type_errors=True</div></td>\n",
       "      <td><div class=\"dictlist\">top_features_vector=store://feature-vectors/fraud-demo-dani/short</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "run_id = project.run(\"main\", arguments={}, dirty=True, watch=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-step-ui-pipeline-view\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6: Test the model endpoint\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that your model is deployed using the pipeline, you can invoke it as usual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-02-15 14:56:50,310 [info] invoking function: {'method': 'POST', 'path': 'http://nuclio-fraud-demo-dani-serving.default-tenant.svc.cluster.local:8080/v2/models/fraud/infer'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'id': 'dbc3b94e-367d-4970-8825-f99ebf76320b',\n",
       " 'model_name': 'fraud',\n",
       " 'outputs': [0]}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define your serving function\n",
    "serving_fn = project.get_function(\"serving\")\n",
    "\n",
    "# Choose an id for your test\n",
    "sample_id = \"C1000148617\"\n",
    "model_inference_path = \"/v2/models/fraud/infer\"\n",
    "\n",
    "# Send our sample ID for predcition\n",
    "serving_fn.invoke(path=model_inference_path, body={\"inputs\": [[sample_id]]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"gs-tutorial-4-done\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Done!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
